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(54) Two-dimensional code recognition processing 



(57) Disclosed are a two-dimensional code recogni- 
tion processing method, a two-dimensional code recog- 
nition processing apparatus and two-dimensional code 
recognition processing program storage medium for 
recognizing a two-dimensional code made of a plurality 
of square cells arranged in accordance with predeter- 
mined layout rules, wherein two-dimensional code rec- 



ognition processing method comprises the steps of rec- 
ognizing code data assigned to the two-dimensional 
code, from externally captured image information, and 
if a plurality of code data are recognized successively 
over time in the recognizing step, or if a plurality of code 
data are recognized from the same image information 
in the recognizing step, then linking the plurality of code 
data to generate expanded code data. 
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D script ion 

BACKGROUND OF THE INVENTION 

5 [0001] The present invention relates to two-dimensional code recognition processing. 

[0002] The so-called bar code system is used extensively in various industrial fields. The system involves encoding 
in bars alphanumeric characters representing types and conditions of goods and articles, and typically attaching such 
codes to the goods and articles so that the codes may be scanned later for retrieval of relevant information (e.g., about 
their types and conditions). 

10 [0003] Fig. 23 shows a so-called one-dimensional bar code. In this example, a one-dimensional bar code label 500 
is made up of a code part 501 and an ID (identification number) part 502. The code part 501 includes a combination 
of different widths of bars (in black) and bar-to-bar spaces (blanks). The ID part 502 indicates a code contained in the 
code part 501. Specifically, the ID part 502 shows as readable information alphanumeric characters that are coded in 
the code part 501 . This type of one-dimensional bar code 500 is read by an optical recognition apparatus called a bar 

is code scanner for recognition. 

[0004] One way of reading the above-described one-dimensional bar code 500 is by use of a CCD video camera. 
As more and more bare are arranged to accommodate growing volumes of coded information, however, it becomes 
increasingly difficult for the CCD video camera to properly read such one-dimensional bar codes 500. 
[0005] The difficulty in capturing large amounts of bar-coded information with the video camera is solved by a two- 

20 dim nsional code system proposed to replace the one-dimensional bar code. As depicted in Fig. 24, various kinds of 
two-dimensional codes 600 have been proposed, each code comprising a code part 601 and an ID part 602. The code 
part 601 has a plurality of black square cells arranged two-dimensional ly according to predetermined layout rules. The 
ID part 602 indicates a code that is contained in the code part 601. 

[0006] Greater quantities of information are represented when the information is coded two-dimensionally than if the 
25 one-dimensional code 500 is used. In other words, it is easier for the video camera to read larger quantities of infor- 
mation if the information is coded two-dimensionally rather than one-dimensionally. 

[0007] However, the two-dimensional code 600 has one disadvantage. That is, when the video camera picks up an 
image of a two-dimensional code 600 for code recognition, it is often difficult to distinguish the code from smear, dirt 
or other distracting images picked up nearby. The result is that code data are often difficult to recognize precisely. 
30 [0008] In addition, only the code data assigned to each single two-dimensional code 600 has been usable in a related 
art, which poses constraints on the quantity of information made available for identification purposes. The limited scope 
of ID information availability in turn has restricted the scope of applications. 

SUMMARY OF THE INVENTION 

35 

[0009] The present invention seeks to provide a method and an apparatus for permitting efficient and accurate rec- 
ognition of expanded code data made of a plurality of two-dimensional codes in combination. 

[0010] According to one aspect of the present invention, there is provided a two-dimensional code recognition 
processing method for recognizing a two-dimensional code made of a plurality of square cells arranged in accordance 
40 with predetermined layout rules, the method including the steps of: recognizing code data assigned to the two-dimen- 
sional code captured from external image information; and linking the plurality of code data to generate expanded code 
data, if a plurality of code data are recognized successively over time in the recognizing step, or if a plurality of code 
data are recognized from the same image information in the recognizing step. 

[001 1] According to another aspect of the invention, there is provided a two-dimensional code recognition processing 
45 apparatus for recognizing a two-dimensional code made of a plurality of square cells arranged in accordance with 
predetermined layout rules, the apparatus comprising: recognizing means for recognizing code data assigned to the 
two-dimensional code captured from external image information; and linking means for linking the plurality of code 
data to generate expanded code data, if a plurality of code data are recognized successively over time by the recog- 
nizing means, or rf a plurality of code data are recognized from the same image information by the recognizing means. 
50 [0012] According to a further aspect of the invention, there is provided a two-dimensional code recognition processing 
program storage medium for storing a two-dimensional code recognition processing program in a manner executable 
by a computer, the two-dimensional code recognition processing program comprising the steps of: recognizing code 
data assigned to the two-dimensional code captured from external image information; and linking the plurality of code 
data to generate expand d code data, if a plurality ol code data ar recogniz dsuccessiv lyov rtimeinther cognizing 
55 step, or if a plurality of code data are recognized from the same image information in th recognizing step. 

[001 3] Through the use of th inventive two-dimensional code recognition processing method, two-dimensional code 
recognition processing apparatus, and two-dimensional code recognition processing program storage medium outlin d 
above, code data assigned to a two-dimensional code are recognized from the externally captured image information. 
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If a plurality of code data are successively recognized over time or if these code data are recognized from the same 
image information, then the code data are linked so as to generate expanded code data. 

[0014] Embodiments of the present invention relate to a two-dimensional code recognition processing method, a 
two-dimensional code recognition processing apparatus, and a two-dimensional code recognition processing program 
5 storage medium for permitting efficient and accurate recognition of expanded code data made of a plurality of suitably 
captured two-dimensional codes in combination. 

[0015] For a better understanding of the present invention reference will now be made, by way of example, to the 
accompanying drawings, in which: 

io Fig. 1 is a perspective view of a portable personal computer with its display part swung open away from its body; 

Fig. 2 is a plan view of the computer in Fig. 1; 

Fig. 3 is a left-hand side view of the computer in Fig. 1 with its display part closed onto the body; 

Fig. 4 is a right-hand side view of the computer in Fig. 1 with its display part swung open 180 degrees relative to 

its body; 

15 Fig. 5 is a front view of the computer in Fig. 3; 

Fig. 6 is a bottom view of the computer in Fig. 4; 

Fig. 7 is a block diagram showing a typical electrical circuit structure of the computer in Fig. 1 ; 
Fig. 8 is a schematic view showing a use example of a personal computer 1; 
Fig. 9 is an explanatory view of two-dimensional code specifications; 
20 Fig. 1 0 is another explanatory view of two-dimensional code specifications; 

Fig. 11 is another explanatory view of two-dimensional code specifications; 
Figs. 12A through 12D are views depicting typical two-dimensional codes; 

Figs. 13A and 13B are flowcharts showing steps constituting a two-dimensional code recognition process: 

Fig. 14 is an explanatory view indicating threshold value settings; 
2$ Fig. 15 is a schematic view depicting how black pixel contiguous regions are labeled; 

Fig. 16 is a flowchart showing steps constituting a logo mark cell part detection process; 

Fig. 17 is an explanatory view illustrating how sides AT and BT are obtained; 

Fig. 18 is a flowchart showing steps constituting a code part detection process; 

Fig. 1 9 is a flowchart showing steps constituting a code data verification process; 
30 Fig. 20 is a flowchart showing steps constituting a process of recognizing a plurality of two-dimensional codes; 

Fig. 21 is a flowchart showing steps constituting another process of recognizing a plurality of two-dimensional 

codes; 

Fig. 22 is an explanatory view of an object bearing a plurality of two-dimensional codes; 
Fig. 23 is a schematic view of a related art one-dimensional bar code; and 
35 Fig. 24 is a schematic view of a related art two-dimensional code. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0016] Preferred embodiments of this invention will now described with reference to the accompanying drawings. 

40 [0017] Figs. 1 through 6 show typical structures of a portable personal computer 1. 

[0018] The personal computer 1 is a mini-notebook type personal computer that primarily comprises a body 2 and 
a display part 3 attached swingingly to the body 2. Fig. 1 is a perspective view of the computer with the display part 3 
swung open away from the body 2. Fig. 2 is a plan view of the computer in Fig. 1 . Fig. 3 is a left-hand side view of the 
computer with the display part 3 swung shut onto the body 2. Fig. 4 is a right-hand side view of the computer with the 

45 display part 3 swung open 180 degrees relative to the body 2. Fig. 5 is a front view of the computer in Fig. 3. Fig. 6 is 
a bottom view of the computer in Fig. 4. 

[0019] The front face of the body 2 comprises a keyboard 4 and a stick type pointing device 5. The keyboard 4 is 
used to input characters, symbols, etc., and the stick type pointing device 5 is used to move a mouse cursor. Also 
furnished on the front face of the body is a speaker 8 for sound output along with a shutter button 1 0 operated to shoot 

50 an image using the CCD video camera 23 provided on the display part 3. 

[0020] A pawl 1 3 is provided at the upper end of the display part 3. As shown in Fig. 3, with the display part 3 closed 
onto the body 2, the pawl 1 3 hooks into a hole 6 in the body 2. At the front of the body 2 is a slide lever 7 furnished in 
a parallelly slidable fashion. The slide lever 7 is used to lock and unlock the pawl 1 3 engaged with the hole 6. With the 
pawl 1 3 unlocked, the display part 3 may be swung open away from the body 2. Adjacent to the pawl 1 3 is a microphone 

55 24 which, as depicted in Fig. 6, may pick up sound from both the front and the back side of the body 2. 

[0021] The front of the body 2 further comprises a programmable power key (PPK) 9. An air outl 1 11 is provided on 
. the right-hand side of the body 2, as shown in Fig. 4. At the lower end on the front of the body 2 is an air inlet 14 as 
depicted in Fig. 5. To the further right of the air outlet 11 is a slot 12 to accommodate a PCMCIA (Personal Computer 
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Memory Card International Association) card (called a PC card). 

[0022] An LCD (liquid crystal display) 21 tor displaying images is provided on the front of the display part 3. At the 
upper end of the LCD 21 is an image pickup part 22 mounted rotatably on the display part 3. More specifically, the 
image pickup part 22 is rotatable to any position within a range of 180 degrees in the same direction as the LCD 21 
s and in the opposite direction thereof (i.e., toward the back). The image pickup part 22 is furnished with the CCD video 
camera 23. 

[0023] At the lower end of the display part 3 on the body side is a group of lamps including a power lamp PL, a battery 
lamp BL, a message lamp ML and other lamps formed of LEDs. Reference numeral 40 in Fig. 3 denotes a power switch 
furnished on the left-hand side of the body 2, and reference numeral 25 in Fig. 5 represents an adjusting ring used to 
10 adjust the focus of the CCD video camera 23. Reference numeral 26 in Fig. 6 stands for a cover that conceals an 
opening through which an additional memory is installed into the body 2, and reference numeral 41 denotes a hole 
through which to insert a pin to unlock the pawl of the cover 26. 

[0024] Fig. 7 illustrates an internal structure of the personal computer 1. As shown in Fig. 7, an internal bus 51 is 
connected to a CPU (central processing unit) 52, a PC card 53 inserted as needed, a RAM (random access memory) 
is 54, and a graphic chip 81. The internal bus 51 is coupled to an external bus 55. The external bus 55, for its part, is 
connected to a hard disk drive (HDD) 56, an I/O (input/output) controller 57, a keyboard controller 58, a stick type 
pointing device controller 59, a sound chip 60, an LCD controller 83, and a modem 50. 

[0025] The CPU 52 is a controller that controls diverse computer functions. The PC card 53 is installed as needed 
when an optional function is to be added. 

20 [0026] Image data captured by the CCD video camera 23 are forwarded to a processing part 82 for processing. The 
image data processed by the processing part 82 are input to the graphic chip 81 connected to the internal bus 51 . The 
graphic chip 81 stores the input image data into an internal VRAM 81 A, and retrieves the data from the memory as 
needed for output to the LCD controller 83. Given the image data from the graphic chip 81 , the LCD controller 83 
outputs the data to the LCD 21 for display. Back lights 84 are provided to illuminate the LCD 21 from the back. 

2S [0027] When the personal computer 1 completes boot-up, an electronic mail program (an application program) 54A, 
an auto pilot program (another application program) 54B and the OS (operating program) 54C are transferred from 
the HDD 56 to the RAM 54 and retained therein. 

[0028] The electronic mail program 54A is a program that exchanges correspondence with an external entity using 

a communication line such as a telephone line by way of a network. A received mail acquisition function is specifically 
30 included in the electronic mail program 54A. The received mail acquisition function checks a mail server 93 to see if 

a mail box 93A therein contains any e-mail addressed to this program (i.e.. to the user). If any such mail is found in 

the mail box 93A, the received mail acquisition function carries out a suitable process to acquire that mail. 

[0029] The auto pilot program 54B is a program that starts up and carries out a plurality of predetermined processes 

(or programs) in a predetermined sequence. 
35 [0030] The OS (operating system) 54C controls basic computer functions. Typical operating systems are Windows 

95 (registered trademark), Windows 98 (registered trademark) and the like. 

[0031] The hard disk drive (HDD) 56 connected to the external bus 55 side contains the electronic mail program 
56A, the auto pilot program 56B ; the OS (operating system, or basic program software) 56C, and a two-dimensional 
code recognition program 56D. 
40 [0032] The two-dimensional code recognition program 56D in operation detects a logo mark cell part 301 of a logo 
mark part 201 and a code part 202, of a two-dimensional code from image data held in the VRAM 81 A of the graphic 
chip 81 . The two-dimensional code recognition program 56D then recognizes code data of the code part 202 so as to 
retrieve relevant coded information therefrom. 

[0033] The I/O controller 57 has a microcontroller 61 equipped therewith an I/O interface 62. The microcontroller 61 
45 js constituted by the I/O interface 62, a CPU 63, a RAM 64 and a ROM 69 which are interconnected to each other. The 
RAM 64 includes a key input status register 65, an LED (light-emitting diode) control register 66, a set time register 
67, and a register 68. The set time register 67 is used to start the operation of a start sequence controller 76 when a 
time preset by the user (i.e., starting condition) is reached. The register 68 holds a correspondence between a preset 
combination of operation keys (starting condition) on the one hand and an application program to be started on the 
50 other hand. When the user inputs the preset combination of operation keys, the corresponding application program 
(e.g., electronic mail program) is started. 

[0034] When the fingertip-operated programmable power key (PPK) 9 is pushed, the key input status register 65 
gets and retains an operation key flag. The LED control register 66 is used to control the illumination of the message 
lamp ML indicating that boot-up status of an application program (e.g., electronic mail program) which is held in the 
55 register 68. A desir d tim of a day may be set to the set time register 67. 

[0035] The microcontroller 61 is connect d to a backup battery 74. The battery 74 allows values of the r gisters 65, 
66 and 67 to be retained when power to the body 2 is turned off. 

[0036] The ROM 69 in the microcontroller 61 contains in advance a wake-up program 70, a key input monitoring 
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program 71, and an LED control program 72 in advance. The ROM 69 is illustratively composed of an EEPROM 
(electrically erasable and programmabl read only memory). The EEPROM is also called a flash memory. The micro- 
controller 61 is connected to an RTC (real-time clock) 75 that continuously keeps the current time. 
[0037] The wake-up program 70 in the ROM 69 is a program that checks to see if a preset time in the set time register 

5 67 is reached on the basis of real-time clock data from the RTC 75. When the preset time is reached, the wake-up 
program 70 starts up a predetermined process (or program). The key input monitoring program 71 continuously mon- 
itors whether the PPK 9 is pushed by the user. The LED control program 72 controls the lighting of the message lamp ML. 
[0038] Furthermore, the ROM 69 contains a BIOS (basic input/output system) 73. The BIOS is a software program 
that controls exchanges of data (input and output) between the OS or application software on the one hand and pe- 

10 ripheral devices (e.g., display part, keyboard, hard disk drive) on the other hand. 

[0039] The keyboard controller 58 connected to the external bus 55 controls input from the keyboard 4. The stick 
type pointing device controller 59 controls input from the stick type pointing device 5. 

[0040] : The sound chip 60 receives input from the microphone 24, or supplies sound signals to the built-in speaker B. 
[0041] The modem 50 permits connection to a communication network 92 such as the Internet and to the mail server 

is 93 through a public telephone line 90 and an Internet service provider 91 . 

[0042] The power switch 40 is operated to turn on and off the power supply. A half-push switch 85 is activated when 
the shutter button 10 is half-pushed. A full-push switch 86 is turned on when the shutter button 10 is fully pushed. A 
reverse switch 87 is turned on when the image pickup part 22 is rotated by 180 degrees (i.e., when the CCD video 
camera 23 is rotated into a direction suitable for picking up an image on the opposite side of the LCD 21 ). 

20 [0043] Fig. 8 shows a personal computer 1 . In this example, an object 1 00 is illustratively a business card-like object. 
A two-dimensional code label 101 is attached, including printing, thereon. 

[0044] As depicted in Fig. 9, the two-dimensional code label 101 is made of a plurality of square cells (black portions) 
arranged two-dimensional ly in accordance with predetermined layout rules. Such a layout pattern of the cells represents 
predetermined coded form alphanumeric characters, such as a number 200 in this example. 

25 [0045] The two-dimensional code will now be described in more detail. The two-dimensional code 1 01 comprises a 
logo mark cell part 201 and a code part 202. The two parts as a whole are formed in an area consisting of seven blocks 
(each block represents a rectangular area of a single square cell) in the X-axis direction and 9.5 blocks in the Y-axis 
direction. That area is defined as the 7 X 9.5 block area, and like definition will also apply hereunder. 
[0046] As illustrated in Fig. 10, the logo mark part 201 is made of a logo mark eel! part 301 and a non-cell part 302. 

30 The logo mark cell part 301 constitutes a 7 X 1 .5 block rectangular area which, when printed, representing a hollowed 
logo mark, characters, numerals and other hollowed readable character information associated with the two-dimen- 
sional code. Illustratively. "CyberCode" attached to the two-dimensional code system forms a typical logo mark. Such 
information signifying what the two-dimensional code in question means is given in hollowed characters constituting 
a human-readable mark. 

35 [0047] The logo mark part 301 is not limited to expressing only logo marks. The part may illustratively represent the 
name of the corporation of the applicant of the present invention which developed the two-dimensional code system 
and for which this applicant works. The logo mark part may also express a URL (Uniform Resource Locator) indicating 
where information resources are located in connection with the code scheme of two-dimensional code 101. URL is a 
method for uniquely designating locations where files and other information resources are stored in a distributed manner 

40 on the Internet. 

[0048] When the rectangular logo mark cell part 301 indicates in atypical logo mark, format what the two-dimensional 
code signifies as described, those who are interested in the logo mark can gain access to and grasp information about 
the logo mark at relevant home pages and other locations on the Internet. Illustratively, the following description is 
available at the time of submitting this application from the homepage offered by this applicant at (URL): 

45 http://www.sony.co.jp/sd/Prc>ductsPark/Consumer/PCOIvVPCGC1CAT/cybercode.html 

[0049] "What we call "CyberCode" is Sony's unique two-dimensional code system that offers about 16.77 million 
different patterns (in 24 bits). Of these patterns, about one million patterns (in 20 bits) may be registered as desired 
for program start-up purposes. The remaining code patterns are reserved for future service expansion. "CyberCode" 
works as an index to what is represented by the code in question, the index allowing relevant information to be retrieved 

50 from computer storage. When a user starts a program through a new interface feature "CyberCode Finder, B the user 
finds that the corresponding information leaps from the object having the "CyberCode" onto the computer screen." 
[0050] In the logo mark cell part 301 , as shown in Fig. 1 0, a 7 x 1 block area on the side of the code part 202 includes 
the non-cell part 302 that contains no cell. The major axis of the logo mark cell part 301 is called a side A (in the X- 
axis direction in Fig. 9) and its minor axis is called a side B (in the Y-axis direction in Fig. 9). 

55 [0051] As shown in Fig. 11, the cod part 202 has atop left corner cell and a top right corner cell. With the logo mark 
cell part 301 oriented downward, the top left corner cell is located seven blocks above from the leftmost edge of th 
logo mark cell part 301 ; the right top corner cell is seven blocks above from the rightmost edge of the logo mark cell 
part 301. In addition, a bottom left corner cell is located one block above from the leftmost edge of the logo mark cell 
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pari 301 , and a bottom right com r cell is one block above from the rightmost dge of the logo mark cell part 301 . The 
top left, top right, bottom left and bottom right corner cells are collectively referred to as the corner c lis unless they 
.need to be distinguished specifically from one another. 

[0052] A three-block area contiguously surrounding each corner cell contains no cells. In other words, the corner 
cells are defined as the cells located as described relative to the logo mark cell part 301 , each cell having no cells in 
the three-block area therearound. 

[0053] If one block is assumed to correspond to one bit, the code part 202 is supposed to represent 49 bits of infor- 
mation (7X7 blocks = 49 blocks) in coded form. However, each corner cell (1 block each) and the three-block area 
therearound do not constitute data as mentioned above. That is, a total of 16 blocks in the code part 202 are not used 
for making up data. Of the remaining 33 blocks (33 bits), nine blocks (9 bits) form check data for checking to see if 
given code data are correct. Thus the code part 202 has actually 24 bits of information in coded form. 
[0054] Figs. 12A through 12D show typical two-dimensional codes. Code parts 202 of two-dimensional code 101 in 
Fig. 12A indicate code data representing ID numbers 200, 201, 202, 203, 1500, 1501, 1502 and 1503 in coded form 
are printed. In an ID part 203 under each logo mark cell part 301 is a printed number corresponding to a coded ID 
number. 

[0055] In Fig. 12B, each logo mark cell part 301 of the two-dimensional code 101 has a logo mark printed as hollowed 
characters. Fig. 12C omits a number corresponding to the ID number in the ID part 203 of each two-dimensional code 
label 101. In Fig. 12D, each logo mark cell part 301 has a number corresponding to the ID number, printed as hollowed 
characters. The logo mark is printed in black in the ID part 203. The code parts 202 of the two-dimensional code 101 
in Figs. 1 2A through 1 2D are all prepared according to the same specifications. 

[0056] The above-mentioned logo mark cell part 301 is constituted by a rectangular area having a predetermined 
aspect ratio and including a black pixel contiguous region made of a plurality of contiguously arranged black pixels, as 
will be d scribed later. The logo mark cell part 301 further comprises readable information (as hollowed) associated 
with th two-dimensional code 101 . When the process of two-dimensional code 101 recognition is first started, a rec- 
tangular logo mark cell part 301 having a predetermined aspect ratio is detected in the initial stage. The detected logo 
mark cell part 301 is used as a reference for detecting and recognizing other cells that may exist in a predetermined 
search range. That is, the logo mark cell part 301 is not merely provided as a readable logo mark format indicating the 
meaning of the two-dimensional code 101; the logo mark cell part 301 also offers a reference function for use in the 
two-dimensional code recognition process. 

[0057] As described, the logo mark cell part 301 has not only the reference-indicating function but also the function 
of displaying logo marks and characters, as shown in Figs. 1 2B through 1 2D, thereby enabling to provide both reference 
information necessary for the recognition process and human-readable information at once in the smallest possible 
area that is occupied. 

[0058] Returning to Fig. 8, the personal computer 1 is capable of picking up the object 100 and two-dimensional 
code 101 using the CCD video camera 23. The computer 1 proceeds to recognize code data of the captured two- 
dimensional code 101. 

[0059] A process of recognizing a two-dimensional code will now be described with reference to the flowchart of Fig. 
13. 

[0060] When the CCD video camera 23 picks up a two-dimensional code 101 and acquires a single-frame image 
data therefrom, the data are processed by the processing part 82 and placed into the VRAM 81 A of the graphic chip 
81. .While being in that state, the two-dimensional code recognition program 56D is started from the HDD 56. In step 
S1 , the CPU 52 initializes a counter value Y to 1 for counting a threshold value representing a brightness level. In this 
example, as shown in Fig. 1 4, the threshold value may be set to one of five levels. More specifically, the counter setting 
may range descendingly from threshold value A, a maximum value corresponding to set No. 1, to a threshold value 
E, a minimum value corresponding to set No. 5. 

[0061] In step S2, the CPU 52 performs binarization on the image data held in the VRAM 81 A in accordance with 
the threshold value that is set on the counter T. Each pixel greater than the threshold value in brightness is coded its 
pixel value as "0". The "0" coded pixel is shown white when displayed. In the description that follows, each pixel whose 
pixel value is coded as "0" is called a white pixel. 

[0062] On the other hand, each pixel less than the threshold value in brightness is coded its pixel value as "1 ". The 
"I" coded pixel is shown black when displayed. In the description that follows, each pixel whose pixel value is coded 
as a 1 D is called a black pixel. 

[0063] In step S3, as shown in Fig. 15, the CPU 52 successively numbers (labels), from top left to bottom right, 
r gions ach compos d of continuous black pix Is. 

[0064] In step S4, th CPU 52 obtains a total number M of black pixel contiguous r gions, and ch cks to see if the 
obtained total number M is at least 257 or greater. If the value M is found to be at least 257 or greater, then the CPU 
52 judges that the image frame currently stored in the VRAM 81 A is not fit for subsequent processing, step S4 is 
followed by step S5. Images unsuitable for subsequent proc ssing are appreciably grainy images known as dither 
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images. Attempts to recognize such imag s forcibly will overload the CPU doing the necessary computations. In such 
cases, recognition of the dither image is skipped and step S5 is reached. 

[0065] In step S5, the CPU 52 checks to see if the value on the counter T is equal to a set number N of the threshold 
value (i.e., i = 5). If the counter value is judged to be other than five, then step S6 is reached and the counter "i" is 
5 incremented by 1. Step S6 is followed by. step S2. In step S2, the CPU 52 again performs binarization on the image 
data held in the VRAM 51 A in accordance with the threshold value whose set number on the counter Y has been 
increased by 1 . 

[0066] As described, the threshold value A represented by set No. 1 is the maximum threshold value in this example. 
When the maximum value A is taken as the threshold value for binarization, a large number of pixels constituting the 

io image data have a brightness level relatively lower than the value A. As a result, the greater part of the pixels are 
recognized as black pixels thereby increasing the total number M of black pixel contiguous regions. If the set number 
on the counter Y is incremented by 1 in step S5, and the next-highest threshold value comes into effect as the threshold 
for the next binarization. This in turn reduces the number of pixels that are regarded as black pixels. 
[0067] - As described, when a relatively large threshold value representing a high level of reference brightness is 

is initially used for binarization, the binarization process is carried out so that the frame as a whole becomes fit for a 
relatively bright image. When the threshold value is reduced progressively to represent lower levels of brightness for 
binarization, the binarization process is performed so that the entire frame becomes fit for relatively dark images. 
Because five alternative threshold levels are provided and because the threshold value is set for one alternative level 
to another, highly accurate binary data may be generated consistently regardless of brightness fluctuations in the entire 

20 image frame. 

[0068] If the CPU 52 judges that "i = 5 B in step S5, i.e., if none of the threshold values A through E is relevant to 
generating a suitable total number M of black pixel contiguous regions, then the CPU 52 judges the pixel data of the 
current frame to exclude any two-dimensional code, and terminates the process. 

[0069] If the CPU 52 judges the total number M of black pixel contiguous regions to be less than 257, then step S7 
25 is reached. 

[0070] In step S7, the CPU 52 judges to see if a certain display location on the LCD 21 of the two-dimensional code 
101 recognized in the previously executed two-dimensional code recognition process, such as a center point of the 
logo mark cell part 301, is stored in the RAM 54 or in other suitable memory, if such a location is found to be stored, 
step SB is reached in which the stored point is set as a start point for a bgo mark part detection process. If the CPU 

30 52 judges in step S7 that no location of the two-dimensional code 101 has been stored, step S9 is reached. In step 
S9, the.CPU 52 sets as the start point a display center point of 320 x 240 pixels on the LCD 21 (e.g., the pixel that 
locates 160th on the X-axis and 120th on the Y-axis). The setting permits efficient detection of a logo mark cell part 301 . 
[0071] In step S10, the logo mark part detection process is carried out. Detailed steps constituting the logo mark cell 
part detection process in step S10 are described below with reference to the flowchart in Fig. 16. 

35 [0072] In step S21 , the CPU 52 initializes the counter "j" to 1 for counting the total number M of black pixel contiguous 
regions. In step S22, the CPU 52 searches for black pixel contiguous regions throughout the screen on the LCD 21 in 
counterclockwise spiral fashion from the start point established in steps S8 and S9 of Fig. 1 3. The black pixel contiguous 
region detected first is selected as a logo mark cell part candidate region. 

[0073] In step S23, the CPU 52 determines sides AT and BT corresponding to the sides A and B of a logo mark cell 
40 part 301 shown in Fig. 10, selected as the logo mark cell part candidate region in step S22. As illustrated in Fig. 17, 
the side AT is the longer of two line segments, one formed by projection onto the X-axis of the image of the black pixel 
contiguous region on the LCD 21 (X-axis projection line), the other formed by like projection of the same region onto 
the Y-axis (Y-axis projection line). The side BT is the shorter of the two projection lines. 

[0074] In step S24, the CPU 52 judges to see if the side BT corresponding to the minor axis is made up of fewer 
45 than 20 pixels. Illustratively, if a black pixel contiguous region whose minor axis (side B) is composed of fewer than 20 
pixels is a logo mark cell part 301 , then the side BT has a still smaller number of pixels because the ratio of a side of 
each block to the side B is 1 to 1.5 as shown in Fig. 9. In that case, the smallest cells (in 1 x 1 block area) become 
too small to be displayed properly on the LCD 21 . Thus if the side BT is judged to have fewer than 20 pixels in step 
S24, the black pixel contiguous region selected this time in step 522 is judged to be other than a logo mark cell part 
so 301 . Step S24 is then followed by step S25 in which the CPU 52 judges to see if a counter °j B has a value that is equal 
to the total number M of black pixel contiguous regions (j = M). If the value on the counter n j" is not judged to be equal 
to the total number M, then step S26 is reached in which the counter "j D is incremented by 1 . Step S26 is followed by 
step S22 in which the CPU 52 regards the next-detected black pixel contiguous region as the next logo mark cell part 
candidate region, and proceeds to carry out the subsequent steps. 
55 [0075] If in step S24 the CPU 52 judges that th side BT of the logo mark cell part candidal region select d in step 
S22 includes equal to or at least 20 pixels, then the CPU 52 goes to step S27, if determined so. In step S27, the CPU 
52 judges to see if the side AT of the logo mark cell part candidate region comprises more than 300 pixels. Illustratively, 
if a black pixel contiguous region whose major axis (side A) comprises more than 300 pixels is a logo mark cell part 
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301 , th n each sid of a block becomes too long to be displayed properly because the ratio of one side of the block 
to the side A is one to seven as depicted in Fig. 9. That is, as shown in Fig. 11 , the top left and top right corner cells 
located seven blocks away from the logo mark cell part 301 will not appear on the LCD 21 . Thus if the side AT is judged 
to have more than 300 pixels in step S27, the black pixel contiguous region selected this time in step S22 is judged to 

s be other than a logo mark cell part 301. In that case, step S24 is followed by step S25. 

[0076] If in step S27 the CPU 52 judges that the side AT of the logo mark cell part candidate region selected in step 
S22 does not comprise more than 300 pixels (i.e., region made up of 300 pixels at most), then step S28 is reached. 
In step S28, the CPU 52 judges to see if the total number of black pixels in the logo mark cell part candidate region is 
at least 20 or greater and less than 1500. If the result of the check in step S28 is affirmative, step S29 is reached. 

10 [0077] If the total number M ol black pixels in the logo mark cell part candidate region is judged in step S28 to be 
fewer than 20 or at least 1500 or greater, then step S25 is reached. When the total number of black pixels is less than 
20, the same problem arises as that occurred when the side BT had fewer than 20 pixels in step S24. Where the total 
number of black pixels is at least 1500 or greater, the same problem develops as that incurred when the side AT is 
found to have more than 300 pixels in step S27. In any case, there is only a limited possibility that the candidate region 

is is a logo mark cell part 301 . 

[0078] In step S29, the CPU 52 judges the fitness of the logo mark cell part candidate region selected in step S22. 
If the fitness of the region is recognized, step S30 is reached. Specifically, the logo mark cell part candidate region is 
judged to be fit if the level of fitness calculated by use of expression (1) below is at least 0.2 or greater. 

mness = Jb 2 +(a-c) 2 Jb 2 + (a-c) 2 (ex 1} 

((a^Ma-c)x-^ 

Vb +(a-c) vb +(a-c) 

25 

[0079] In the expression (1 ) above, constants °a n , n c° and "b/2 B are obtained from expressions (2), (3) and (4) below 
r spectively. These constants are a two-dimensional moment each as part of moment-related characteristics. 



30 



35 



40 



45 



55 



M<2,0) - Si 3 f u< „ 



u " Jl (ex. 2) 



M(0,2) - Z j 2 f fi 2} 

31 (ex. 3) 



M<1,1) - 2 ijf u . 3 , 



31 (ex. 4) 



[0080] In the expressions (2), (3) and (4) above, a function f(i, j) provides 1 if the pixel determined by the X coordinate 
V and Y coordinate "j" on the LCD 21 is black, and provides 0 if the pixel in question is white. 

[0081] In step S30, the CPU 52 calculates by use of expression (5) below the ratio of major axis to minor axis for 
so the logo mark cell part candidate region judged to be fit in step S29. A check is made to see if the calculated ratio v is 
at I ast 2.0 and 25 at most. 

v _ V(a+c)+Jb 2 +(a-c) 2 ^ ex ^ 

[0082] If the calculated ratio is judged to be between 2.0 and 25 inclusive, step S31 is reached. In step S31 . the CPU 
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52 establishes (assumes) the logo mark cell part candidate region selected in step S22 as a logo mark cell part 301 . 
Illustratively, the number denoting the black pixel contiguous region constituting the logo mark cell part candidate region 
is placed into the RAM 54. When the logo mark cell part 301 is detected in the above-described manner, the process 
is terminated. 

s [0083] On the other hand, when logo mark cell part candidate region selected this time in step S22 is judged to be 
other than a logo mark cell part 301, that is, when in step S28 the total number of black pixels was judged to be less 
than 20 or at least 1 500 or greater: when in step S29 the logo mark cell part candidate region was not judged to have 
the necessary fitness; or when in step S30 the ratio of side AT to side BT was less than 2.0 or at least 25 or greater. 
In any of -the above cases, the logo mark cell candidate region established in S22 is determined not to be the logo 

10 mark cell part 301 , and step S25 is reached. The CPU 52 thereupon regards the next-detected black pixel contiguous 
region as the next logo mark cell part candidate region, and proceeds to carry out the subsequent steps. 
[0084] If the value on the counter "j" is judged to be equal to the total number M of black pixel contiguous regions (j 
= M) in step S25, step S31 is bypassed with no logo mark cell part 301 established, and the process is then brought 
to an end: That is, the image data (of one frame) subjected to the current two-dimensional code recognition process 

is are judged to exclude any logo mark cell part 301 . 

[0085] When the logo mark cell pan detection process is terminated as described above, step S11 in Fig. 13 is 
reached next. 

[0086] In step S11, a judgement is made to see if a logo mark cell part 301 was detected in step S10. If the logo 
mark cell part 301 is judged to have been detected, step S12 is reached in which a code part detection process is 
20 carried out. Detailed steps constituting the code part detection process in step S1 2 are described below with reference 
to the flowchart in Fig. 18. 

[0087] In step S41 , the CPU 52 initializes the counter "j" to 1 for counting the total number M of black pixel contiguous 
regions detected in step S3 of Fig. 13. In step S42, the CPU 52 selects as a top left corner cell candidate region a 
black pixel contiguous region whose number corresponds to the counter value. 
25 [0088] In step S43, the CPU 52 judges to see if the length ratio of side AT to side BT determined as shown in Fig. 
17 for the top left corner cell candidate region selected in step S42 is three-fold at most. If the result of the judgement 
in step S43 is affirmative, step S44 is reached. 

[0089] In step S44, the CPU 52 judges if the top left comer cell candidate region selected in step S42 exists within 
a range of search set beforehand for the logo mark cell part 301 detected in step S10 of Fig. 13. If the region in question 
30 is found within the range, step S45 is reached. In step S45, the top left corner cell candidate region is established 
(assumed) as the top left corner cell. 

[0090] If the length ratio of side AT to side BT is judged to be greater than three-fold in step S43, or if the candidate 
region is not found in the range of search in step S44, then the black pixel contiguous region selected in step S42 is 
judged to be other than the top left comer cell and step S46 is reached. In step S46, a judgment is made to see if the 

35 value on the counter "j" is equal to the total number M of black pixel contiguous regions (j = M). If the counter value "j" 
is not judged to be equal to the total number M, then step S47 is reached in which the counter p j D is incremented by 1 . 
Step S47 is then followed by step S42. In step S42, the black pixel contiguous region denoted by the next number is 
regarded as the next top left corner cell candidate region, and the subsequent steps are carried out. 
[0091] When the top left corner cell is established in step S45, step S48 is reached. In stepS48, the CPU 52 initializes 

40 another counter V to 2 for counting the number denoting the current black pixel contiguous region. In step S49, the 
CPU 52 selects as a top right corner cell candidate region a black pixel contiguous region whose number corresponds 
to the value on the counter V. 

[0092] In step S50, the CPU 52 calculates the area ratio of the top left corner cell (whose area is defined by the 
number of pixels therein) set in step S45, to the top right corner cell candidate region (its area defined by its pixel count) 
45 selected in step S49. If the ratio (in terms of area) is judged to be six-fold at most, step S51 is reached. 
[0093] In step S51, the CPU 52 judges if the following expressions (6) and (7) are satisfied: 

(S1/D2)<= 900 (ex.6) 

so 

(S2/D2) < = 900 (ex. 7) 

wh re, S1 stands for the area (number of pixels) of the top left corner cell established in S45, S2 for the area (number 
ss of pixels) of the top right corner cell candidate region selected in step S49, and D for the distance calculated between 
the center point of the top left corner cell and the center point of the top right corner cell candidate region. 
[0094] If the expressions (6) and (7) are judged satisfied in step S51 , step S52 is reached. In step S52, the CPU 52 
establishes (assumes) as the top right comer cell the top right comer cell candidate region selected in step S49. 
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[0095] With the top right corner cell established (assumed) in step S52, st p S55 is reached. In step S55, the CPU 
52 performs affine transformation whereby the region formed by the top left comer cell set in step S45, by the top right 
corner cell established in step S52, and by the logo mark cell part 301 set in step S10 of Fig. 1 3 is turned into a region 
of seven blocks long in the X-axis direction and 9.5 blocks long in the Y-axis direction on the screen of the LCD 21. 
5 The length of one side on each block is calculated on the basis of the side AT or BT of the logo mark cell part 301 
established in step S10. 

[0096] In step S56, the CPU 52 erases from the image transformed in step S55 the logo mark cell part 301 set in 
step S10 of Fig. 13 and a region corresponding to a non-cell part 302 paired with the part 301 (7 x 2.5 block region). 
Inside the 7X7 block region resulting from the erasure, the black pixel contiguous regions are mapped as cells whereby 
io a code map is created. 

[0097] In step S57, the CPU 52 detects a square cell from among the cells making up the code map prepared in 
step S56, and judges to see if a three-block area contiguously surrounding the detected cell is composed of white 
pixels. If the result of the judgement in step S57 is affirmative, step S5B is reached. In step S58, the CPU 52 establishes 
(assumes) the code map created in step S56 as a code part 202 of the two-dimensional code 101. When the code 

is part 202 is detected in the manner described, the process comes to an end. 

[0098] When the area ratio was judged to be greater than six-fold in step S50, when the expressions (6) and (7) 
were not judged satisfied in step S51, or when white pixels were not found in the three-block area contiguously sur- 
rounding the square cell in step 557, then step S53 is reached. In step S53, a judgement is made to see if the value 
on the counter "k" is equal to the total number M of black pixel contiguous regions (k = M). If the value on the counter 

20 "k" is not judged to be equal to the total number M, step S54 is reached in which the counter M k B is incremented by 1. 
Step S54 is followed by step S49 in which the next-numbered black pixel contiguous region is selected as the next top 
right corner cell candidate region, and the subsequent steps are repeated. 

[0099] If in step S53 the value on the counter "k" is judged to be equal to the total number M of black pixel contiguous 
r gions, step S46 is reached. In step S46, a judgement is made to see if the value on the counter "j" is equal to the 

2S total number M. If the value on the counter "j" is not judged to be equal to the total number M, step S47 is reached in 
which the counter "j" is incremented by 1. Step S47 is followed by step S42 in which the next-numbered black pixel 
contiguous region is selected as the next top left corner cell candidate region, and the subsequent steps are repeated. 
[0100] If in step S46 the value on the counter n j B is judged to be equal to the total number M, then the image currently 
subjected to the two-dimensional code recognition process is judged to exclude any two-dimensional code. The process 

30 is then terminated. 

[0101] When the code part detection process is completed as described above, step S13 of Fig. 13 is reached. In 
step S1 3, a check is made to see if a code part 202 has been detected in step S12. If a code part 202 is judged to 
have been detected, step S14 is reached to carry out code data verification process. Detailed steps constituting the 
code data verification process in step S14 are described below with reference to the flowchart in Fig. 19. 
3S [0102] In step S61 , the CPU 52 initializes a counter "p 8 to 1 for counting the number of times in which a reference 
value, to be computed in steps S63 and S65 below, is shifted right one bit. 

[0103] In step S62, the CPU 52 calculates value of code data and check data based on the code map of the code 
part 202 detected in step S12 of Fig. 1 3. 

[0104] In step S63, the CPU 52 performs exclusive OR on the code data value (a bit stream) computed in step S62 
40 and OxFFFFFF, and regards the resulting value (another bit stream) as a reference value (reference bit stream). In 
step S64, the CPU 52 checks to see if the LSB (least significant bit) of the reference bit stream is set to 1. If the LSB 
is not judged to be 1 , step S65 is reached. 

[0105] In step S65, the CPU 52 performs exclusive OR on the reference value computed in step 563 (reference bit 
stream) and 0x8408, and considers the resulting value (another bit stream) to be a new reference value (reference bit 
45 stream). Step S65 is followed by step S66. 

[0106] If in step S64 the CPU 52 judges that the LSB is set to 1 in the reference value (reference bit stream) calculated 
in step S63, the CPu 52 reaches step S66 by skipping step S65. 

[0107] In step S66, the CPU 52 shifts to right by one bit the reference value (reference bit stream) computed in step 
S63 or S65. Step S66 is followed by step S67 in which the CPU 52 judges if the value on the counter "p" is equal to 
50 24 (p = 24; a predetermined shift count). If the value on the counter "p° is not judged to be equal to 24. step S68 is 
reached in which the counter n p° is incremented by 1 . Step s68 is followed by step S64, and the subsequent steps are 
repeated until the value on the counter °p" is judged to be 24 (p = 24) in step S67. 

[0108] If in step S67 the CPU 52 judges the value on the counter "p" to be 24, step 569 is reached. In step S69, the 
CPU 52 p rforms AND on the bit stream computed in steps S64 through S68 and 0x1 FFF. In step S70, the CPU 52 
55 judges if the value obtained by th AND operation in st p S69 is equal to th value of ch ck data comput d in st p 
S62. If th compar d values are found to be equal in step S70, th code part 202 detected in step S13 of Fig. 13 is 
considered to have a suitable pattern as a two-dimensional code 101. In step S71, the code part 202 of the two- 
dimensional code label 101 is finalized. The code data verification process is then terminated. 
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[01 09] If in step S70 the CPU 52 judges that the value computed In step S69 is not qual to the check data calculated 
in step S62, the CPU 52 terminates the process by skipping step S71. 

[0110] When the code data verification process is completed as described above, step St 5 of Fig. 13 is reached. In 
step SI 5, a check is made to see if the code part 202 was finalized in step S71 of Fig. 1 9. If the code part 202 is judged 
5 to have been finalized, then step S1 6 is reached. In step S1 6, the CPU 52 places the value of code data computed in 
step S62 ol Fig. 19 (i.e., value of the two-dimensional code 101) into the RAM 54 for storage therein. The two-dimen- 
sional code recognition process is then terminated. 

[0111] When in step S11 the logo mark cell part 301 is not judged to have been detected, when in step S1 3 the code 
part 202 is not judged to have been detected, or when in step S1 5 the code part 202 is not judged to have been finalized, 
10 then the image data currently subjected to the two-dimensional code recognition process is judged to exclude any two- 
dimensional code, and the process is terminated. 

[0112] In the manner as described, the logo mark cell part 301 representing attributes of a two-dimensional code 
101 such as a logo mark is used as the reference for the recognition process whereby the code part 202 is detected. 
The feature makes it possible to minimize the area occupied by the two-dimensional code. Because the logo mark cell 
15 part 301 offers not only the reference-indicating function but also the function of displaying a logo mark or characters, 
it is possible to provide both reference information necessary for the recognition process and human-readable infor- 
mation at once in the smallest possible area that Is occupied by the code. 

[0113] Fig. 20 is a flowchart of steps whereby a plurality of two-dimensional codes are read and processes corre- 
sponding to the read codes are carried out successively. In this example, the user prompts the personal computer 1 
20 to recognize a plurality of two-dimensional codes one at a time at suitable intervals. In step S81, the CPU 52 checks 
to see if a two-dimensional code is recognized. The process in step S81 is repeated until a two-dimensional code is 
recognized. Once a two-dimensional code is recognized, step S82 is reached. 

[01 14] In step S82, a check is made to see if a continuity flag bit is 1 . The continuity flag bit is any one of 24 bits that 
make up the code part 202 of each two-dimensional code 202. If the flag is found to be set (i.e., the flag is "1 that 

25 means the two-dimensional code in question constitutes data continued to another two-dimensional code that has 
be n previously recognized or has yet to be recognized. If the flag is not judged to be set (i.e., the flag is "0"), that 
means the corresponding two-dimensional code is either a stand-alone two-dimensional code (a code designating a 
self-contained process based on its own data) or the last of a series of two-dimensional codes. 
[01 1 5] If in step S82 the continuity flag of the recognized two-dimensional code is found to be "1 step S83 is reached. 

30 in step S83, the 23-bit code data of the newly recognized two-dimensional code are linked to the code data made up 
of the previously recognized two-dimensional codes. When linking of the data is completed, step S81 is reached again. 
The subsequent steps are again carried out on the next two-dimensional code. 

[0116] If in step S82 the continuity flag of the recognized two-dimensional code is not judged to be D 1 " (i.e., the flag 
is "O"), step S84 is reached. In step S84, a check is made to see if the 24 bits including the continuity bit and constituting 
35 the recognized two-dimensional code are all "0." If it is not found that the 24 bits are ail "O," step S85 is reached. In 
step S85, the same process as that done in step S83 is carried out. When step S83 or S85 is repeated as described, 
a desired number of bits (23, 46, 69, etc.) are provided. Two-dimensional codes are thus used to constitute code data 
of a large volume for use in complicated processing. 

[0117] With the process of step S85 completed, step S86 is reached in which the linked data are established as the 
40 current bit value and stored into the RAM 54. When the continuity flag is 0 and the remaining 23 bits are not all 0, that 
means the two-dimensional code in question is the last of a series of two-dimensional codes (to be linked). In step 
S86, the linked data are placed into the RAM 54. In step S87, a process corresponding to the stored data is carried out. 
[0118] If in step S84 the 24 bits making up the recognized two-dimensional code are judged to be all "0, 0 step S88 
is reached. In step S88, a check is made to see if there exist code data which have been derived from any previously 
45 recognized two-dimensional code and which are stored as code data to be linked in the RAM 54. If such code data 
are judged to be nonexistent, step S89 is reached. In step S89, the 23 bits besides the continuity bit of the currently 
recognized two-dimensional code are established as all zeros and stored as such in the RAM 54. Step S89 is followed 
by step S87 in which a process corresponding to the stored code data is performed. 

[01 19] If in step S88 the stored code data to be linked are judged to exist, step S90 is reached. In step S90, all stored 
50 code data are cleared. That is, if a two-dimensional code with its 24 bits all set to 0 is used on a stand-alone basis, 
the process assigned to the data is carried out; if such a two-dimensional code is used as one of a series of two- 
dimensional codes to be recognized successively, the code data represented by that two-dimensional code are used 
to clear all code data stored before the two-dimensional code in question was recognized. 

[0120] The steps above are arranged to cancel all previous code data if an erron ous two-dim nsional code has 
55 been recognized. The code recognition process is then started from the b ginning. 

[01 21] Fig. 21 is a flowchart of other steps whereby a plurality of two-dimensional codes are also read as described 
and processes corresponding to the two-dimensional codes thus read are carried out successively. This example, as 
shown in Fig. 22, involves the use of an object 100 having four two-dimensional code labels 101-1 through 101-4 
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attached thereto. When the object 100 with its multiple two-dimensional code labels 101-1 through 101-4 is captured 
by the CCD video camera 23, the personal computer 1 recognizes these codes one by one from the same image 
information following the steps in the flowchart of Fig. 21. 

[0122] In Steps S101 through S116, a two-dimensional code label 101-1 is recognized. The process made up of 
steps S101 through S116 is the same as that of steps S1 through S16 in Fig. 13and thus will not be described further. 
Once the two-dimensional code 101-1 is recognized in steps S 101 through S106, stepsS117is reached. Instep S117, 
a check is made to see if there still exists any area where the presence or absence of a logo mark (i.e. , two-dimensional 
code label 101 ) has yet to be determined. 

[0123] Since the two-dimensional code labels 101-2 through 101-4 have not yet to be detected, step S101 is again 
reached, and the subsequent steps are repeated. When repeated execution of steps S1 01 through S11 7 has exhausted 
all areas in which the presence or absence of a log mark has yet to be determined, step S118 is reached. In step S118, 
the RAM 54 is set with the number of the recognized two-dimensional code labels 101 , the code data assigned to the 
individual two-dimensional code labels 1 01 , and their layout information. 

[0124] In this example, the following values and information are written to the RAM 54: the number of the two-di- 
mensional code labels 101 is four: the two-dimensional code label 101-1 may illustratively have a value of "0x111111° 
and is located top left; the two-dimensional code label 101 -2 may have a value of "0x222222" and is located top right: 
the two-dimensional code label 101 -3 may have a value of "0x333333" and is located bottom left; and the two-dimen- 
sional code label 101-4 may have a value of "0x444444" and is located bottom right. 

[0125] After the four two-dimensional code labels 101-1 through 101-4 have been recognized, they are given the 
first through the fourth priority when sequenced from top left to bottom right according to their layout information. The 
later the sequence of a label, the higher the positions of the bits represented by that label. In the example of Fig. 22, 
the two-dimensional code label 101-1 is provided with the first priority, followed by the label 101-2 with the second, 
label 1 01 -3 with the third, and label 101 -4 with the fourth. Therefore the resulting bit combination of the newly expanded 
cod data is given as "0x4444443333332222221 11111". Code data corresponding to the images of the four two-di- 
mensional code labels 1 01-1 through 101-4 are recognized successively from the images of these codes derived from 
the same image information. The recognized code data are linked so as to generate newly expanded code data (bit 
combination). A process corresponding to the code data thus acquired is carried out by the personal computer 1. 
[0126] When a plurality of two-dimensional codes are recognized over time as ahown in the flowchart of Fig. 20, or 
in space as depicted in the flowchart of Fig. 21, it is possible to let the initially recognized two-dimensional code start 
an application program, allow the second two-dimensional code to set up reference files and options, and permit the 
third and subsequent two-dimensional codes to execute other processes continuously. 

[0127] A computer program designed to perform the above-described processes may be retained on such package 
media as floppy disks, CD-ROMs and DVDs; or on semiconductor memories, magnetic disks and the like where the 
program is stored temporarily or permanently; or on wired and wireless communication media such as local area 
networks, the Internet, digital satellite broadcasting networks; or in diverse communication interfaces such as routers 
and modems for transmitting or receiving the program offered by the foregoing media. Such media, networks, interfaces 
and other measures allow the program to be installed in computers for program execution. The image processing 
program storage medium as mentioned in this specification refers broadly to all such media, networks, interfaces and 
measures. 

[0128] As described and according to the inventive two-dimensional code recognition processing method, two-di- 
mensional code recognition processing apparatus, and two-dimensional code recognition processing program storage 
medium described above, code data assigned to a two-dimensional code are recognized from externally captured 
image information. If a plurality of code data are recognized successively over time or if these code data are recognized 
from the same image information, then the code data are linked so as to generate expanded code data, whereby 
enabling to carry out complicated processes using a plurality of two-dimensional codes. 

[0129] As many apparently different embodiments of this invention may be made without departing from the spirit 
and scope thereof, it is to be understood that the invention is not limited to the specific embodiments thereof except 
as defined in the appended claims. 



Claims 

1 . A two-dimensional code recognition processing method for recognizing a two-dimensional code made of a plurality 
of squar c lis arranged in accordanc with predetermined layout rules, the m thod comprising the st ps of: 

r cognizing code data assigned to said two-dimensional code, from externally captur d image information; and 
linking said plurality of code data to generate expanded code data, if a plurality of code data are recognized 
successively over time in said recognizing step, or if a plurality of code data are recognized from the same 
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image information in said recognizing step. 

A two-dimensional code recognition processing method according to claim 1 , wherein said linking step comprises 
the steps of: 

judging whether said plurality of code data recognized in said recognizing step are related to one another; and 
linking said plurality of code data, if said plurality of code data are judged to be related to one another in said 
judging step. 

A two-dimensional code recognition processing method according to claim 1 , wherein said recognizing step com- 
prises the step of recognizing said code data assigned to said two-dimensional code as well as layout information 
about said two-dimensional code from said externally captured image information; and 

said linking step comprises the step of determining priorities or said plurality of code data based on said 
layout information recognized in said recognizing step in order to link said plurality of code data in accordance with 
the priorities thus determined. 

A two-dimensional code recognition processing method according to claim 1 , wherein said recognizing step com- 
prises the steps of: 

generating binary data from said externally captured image information in accordance with a predetermined 
threshold value; 

detecting a reference cell serving as a reference in recognizing said two-dimensional code based on said 
binary data generated in said binary data generating step; 

detecting corner cells each located in a predetermined search range with respect to said reference cell detected 
in said reference cell detecting step, on the basis of said binary data generated in said binary data generating 
step; and 

detecting code data assigned to said two-dimensional code existing in an area of a code part surrounded by 
said reference cell and by said corner cells on the basis of said binary data generated in said binary data 
generating step. 

A two-dimensional code recognition processing method according to claim 4, wherein said reference cell is made 
of a rectangular black pixel contiguous region which has a predetermined aspect ratio and which includes readable 
information in hollowed letters related to said two-dimensional code; and 

said reference cell detecting step further detects the rectangular reference cell having said predetermined 
aspect ratio and including said readable information. 

A two-dimensional code recognition processing method according to claim 5, wherein said readable information 
is a logo mark attached to a code scheme of said two-dimensional code. 

A two-dimensional code recognition processing apparatus for recognizing a two-dimensional code made of a plu- 
rality of square cells arranged in accordance with predetermined layout rules, said apparatus comprising: 

recognizing means for recognizing code data assigned to said two-dimensional code, from externally captured 
image information; and 

linking means used if a plurality of code data are recognized successively over time by said recognizing means, 
or if a plurality of code data are recognized from the same image information by said recognizing means, said 
linking means thereupon linking said plurality of code data to generate expanded code data. 

A two-dimensional code recognition processing program storage medium for storing a two-dimensional code rec- 
ognition processing program in a manner executable by a computer, said two-dimensional code recognition 
processing program comprising the steps of: 

recognizing code data assigned to said two-dimensional code, from externally captured image information; and 
linking said plurality of code data to generate xpanded code data, if a plurality of cod data ar recognized 
successively over time in said recognizing step, or if a plurality of cod data are recognized from the sam 
image information in said recognizing step. 

A computer program product comprising a program for carrying out the method of any one of claims 1 to 5. 
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